#second-order optimization

Microsoft researchers demonstrate that the Muon optimizer drastically speeds up grokking in Transformer models, enabling faster transition from memorization to generalization compared to AdamW.